Rademacher Penalty Optimization with the Generalized Ramp Loss

where F = {yh(x) : ∀ h ∈ H} and φ(h(x), y) is the loss function of interest. The labeled training sample S = {(xi, yi) | i ∈ {1, ..., n}, xi ∈ R, yi ∈ {±1}} is assumed to be generated from an independent, identically distributed (IID) random process. Given each Rademacher random variable σi ∈ {±1} with uniform odds, a realization of the Rademacher random variables σ amounts to a partition of the training set into two approximately equal-sized subsets with high probability. Reexpressing (1) to reflect this explicitly yields